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ABSTRACT 


Consider  the  standard  linear  program: 

Minimize  £  x  .  - 

subject  to:  A  x  ■ 

x  >_  0 

where  A  is  an  m  x  n  matrix.  The  simplex  algorithm  solves  this  linear 
program  by  moving  from  extreme  point  of  the  feasibility  region  to  a  better 
(in  terms  of  the  objective  function  £  x)  extreme  point  (via  the  pivot 
operation)  until  the  optimal  is  reached.  In  order  to  obtain  a  feel  for 
the  number  of  necessary  iterations,  we  consider  a  simple  probabilistic 
(Markov  chain)  model  as  to  how  the  algorithm  moves  along  the  extreme  points. 
At  first  we  suppose  that  if  at  any  time  the  algorithm  is  at  the  jth  best 
extreme  point  then  after  the  next  pivot  the  resulting  extreme  point  is 
equally  likely  to  be  any  of  the  j  -  1  best.  Under  this  assumption,  we 
show  that  the  time  to  get  from  the  Nth  best  to  the  best  extreme  point  has 
approximately,  for  large  N  ,  a  Poisson  distribution  with  mean  equal  to 
the  logarithm  (base  e)  of  N  .  We  also  consider  a  more  general  probabi¬ 
listic  model  in  which  we  drop  the  uniformity  assumption  and  suppose  that 
when  at  the  jth  best  the  next  one  is  chosen  probabilistically  according  to 
weights  wi  ,  i  ■  1,  ....  j  -1  . 
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A  SIMPLE  HEURISTIC  APPROACH  TO  SIMPLEX  EFFICIENCY 

by 

Sheldon  M.  Ross 


1.  INTRODUCTION 

Consider  the  standard  linear  program: 

Minimize  jc  x 
subject  to:  A  x  ■  ]> 
x  >  0 

where  A  is  an  m  *  n  matrix.  The  simplex  algorithm  solves  this 
linear  program  by  moving  from  extreme  point  of  the  feasibility  region 
to  a  better  (in  terms  of  the  objective  function  £  x)  extreme  point 
(via  the  pivot  operation)  until  the  optimal  is  reached.  As  there  are 
roughly  N  =  (”)  such  extreme  points  it  would  seem  that  this  method 
might  take  many  iterations  but,  surprisingly  to  some,  this  does  not 
appear  to  be  the  case  in  practice. 

In  order  to  obtain  a  feel  for  whether  or  not  the  above  is  sur¬ 
prising,  we  consider  a  simple  probabilistic  (Markov  chain)  model  as  to 
how  the  algorithm  moves  along  the  extreme  points.  At  first  we  suppose 
that  if  at  any  time  the  algorithm  is  at  the  jth  best  extreme  point 
then  after  the  next  pivot  the  resulting  extreme  point  is  equally  likely 
to  be  any  of  the  j  -  1  best.  Under  this  assumption,  we  show  that  the 
time  to  get  from  the  Nth  best  to  the  best  extreme  point  has  approximately, 
for  large  N  ,  a  Poisson  distribution  with  mean  equal  to  the  logarithm 
(base  e)  of  N  .  We  also  consider  a  more  general  probabilistic  model 


in  which  we  drop  the  uniformity  assumption  and  suppose  that  when  at 
the  jth  best  the  next  one  is  chosen  probabilistically  according  to 
weights  wi  ,  i  ■  1,  j  -1  • 


2.  THE  UNIFORM  MARKOV  CHAIN 


Consider  a  Markov  chain  for  which  *  1  and 

PU  ■  rb  •  3  ■ 1 . .  - 1  .  i  >  1 

and  let  T^  denote  the  number  of  transitions  to  get  from  state 
to  state  1  .  Then  TN  can  be  expressed  as 


M-l 

rN-  l  h 

j=»l  J 


where 


'j'lo 


if  the  process  ever  enters  j 
otherwise  . 


Proposition  1: 

1^,  I^_^  are  independent  and 

PUj  -  1}  -  1/j  ,  1<]<N-1  . 


Proof : 

Given  I^+^ ,  ....  1^  let  n  =  min  {i  :  i  >  j  ,  1^  *  1}  . 


I  }  .  m a--  u 

V  j/(n  -  1) 


1/3 


Then 
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Corollary  2: 


N-l 

(i)  E[T]  -  l  1/j 

j-1 


(11)  var  (V  -  Y  J  (l  -  f) 


(iii)  For  N  large,  has  approximately  a  Poisson  distribution 

with  mean  log  N  . 


Proof : 


Parts  (i)  and  (ii)  follow  from  Proposition  1  and  the  representation 


TN  -  l  I  .  Part  (iii) 


follows  from  the  Poisson  limit  theorem  since 


and  so 


N  N  1  N-l 

Jt*  'I  1? 

i  i 


log  N  <  l  1/j  <  1  +  log  (N  -  1) 
1 


N-l 

log  N  X  l  1/j  .1 

j-1 
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3.  APPLICATION  TO  SIMPLEX 

Assuming  that  n  ,  m  and  n  -  m  are  all  large,  we  have  by 
Stirling's  approximation  that 


N 


(n  -  m) 


n+  1/2 
n _ 

n-m+  1/2  m  + 
in 


and  so  letting  c  =  n/m 

log  N  ~  (me  +  1/2)  log  (me)  -  (m(c  -  1)  +  1/2)  log  (m(c  -  1)) 
-  (m  +  1/2)  log  m  -  1/2  log  (2tt) 


or 


log  N  ~  m|c  log  ^  +  log  (c  -  1) 

Now,  as  lim  x  log  (x/x  -  1)  *  1  ,  it  follows  that  when  c  is  large 
x-*® 

log  N  ~m[l  +  log  (c  -  1)]  . 

Thus  for  instance  if  n  *  8000  ,  m  =  1000  ,  then  the  number  of  necessary 
transitions  is  approximately  Poisson  distributed  with  mean 
1000(1  +  log  7)  S  3000  .  As  the  variance  is  equal  to  the  mean,  we  see 
by  the  normal  approximation  to  the  Poisson  that  the  number  of  necessary 
transitions  would  be  roughly  between 

3000  +  2/3000  or  ,  roughly,  3000  +  110 


95  percent  of  the  time. 
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4.  A  WEIGHTED  MARKOV  CHAIN  MODEL 


Suppose  now  that  =  1  and 


p  =  - - - -  j  <  i  -  1 

ij  w  +  ...  +  w^_^  — 


With  this  model  we  are  thus  able  to  give  more  weight  to  those  states 
closest  to  the  one  presently  at  by  letting  w^  increase  in  j  . 
Analogously  with  Proposition  1,  we  have 


Proposition  2: 


jl  if  j  is  ever  visited 

^  (o  otherwise  . 


Then  lv 


,  are  independent  and 


P{I  =  1}  =  — p3 —  ,  1  <  ]  £  N  -  1 


1  wi 


In  addition,  if  T„  *  £  I  .  Then 

1=1  J 


N-l  /  1  \ 

E[TN]  =  \V<[  Wi) 


N-l  W  y  W 

Var  <v '  L  >  1  -f- 

1 hi  K 
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If  for  instance  we  use  polynomial  weights — w 


i 


ja  ,  0  <_  a  <  00  ,  then 


i  1  i* 

-1  i=l 


j 

Jsj*  xadx 


=  i 


ct+1  _ 
a  +  1 


1 


and  so 


Hence 


(a  +  D3 


,  a+l 


-  1 


a  +  l 

i 


N-l 

E[T  ]  ~  f  2_t_i  dx  =  (a  +  1)  log  (N  -  1) 

N  J  X 

1 

and  thus  in  this  case  TN  has,  for  large  N  ,  approximately  a  Poisson 
distribution  with  mean  (a  +  1)  log  N  .  Thus  when  N  *  ,  the  number  of 

transitions  (i.e.,  simplex  pivot  iterations)  is  approximately  Poisson  with 
mean 


(a  + 


l)m  c  log  +  loS  (c  “ 


n/m 


which  when  c  is  large  is  approximately 


(a  +  l)m[l  +  log  (c  -  1)]  . 


REFERENCES 


Dantzig,  G.  B. ,  "Expected  Number  of  Steps  of  the  Simplex  Method  for  a 
Linear  Program  with  a  Convexity  Constraint,"  Systems  Optimization 
Laboratory  Technical  Report  SOL  80-3,  Stanford  University, 

March  1980. 

Liebling,  T.  M. ,  "On  the  Number  of  Iterations  of  the  Simplex  Method," 
Methods  of  Operations  Research,  XVII,  V,  Oberwolfach-Tajung  uber 
Operations  Research,  13-19,  August  1977,  248-264. 

Orden,  A. ,  "A  Step  Towards  Probabilistic  Analysis  of  Simplex  Convergence, 
Mathematical  Programming,  19,  1980,  3-13. 


